Data quality management using business process modeling

ABSTRACT

A business process modeling framework is used for data quality analysis. The modeling framework represents the sources of transactions entering the information processing system, the various tasks within the process that manipulate or transform these transactions, and the data repositories in which the transactions are stored or aggregated. A subset of these tasks is associated as the potential error introduction sources, and the rate and magnitude of various error classes at each such task are probabilistically modeled. This model can be used to predict how changes in transactions volumes and business processes impact data quality at the aggregate level in the data repositories. The model can also account for the presence of error correcting controls and assess how the placement and effectiveness of these controls alter the propagation and aggregation of errors. Optimization techniques are used for the placement of error correcting controls that meet target quality requirements while minimizing the cost of operating these controls. This analysis also contributes to the development of business “dashboards” that allow decision-makers to monitor and react to key performance indicators (KPIs) based on aggregation of the transactions being processed. Data quality estimation in real time provides the accuracy of these KPIs (in terms of the probability that a KPI is above or below a given value), which may condition the action undertaken by the decision-maker.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present application generally relates to modeling and quantitativeanalysis techniques for managing the quality of data and, moreparticularly, to extending a business process model with constructs toidentify the sources data whose quality is of interest, the datatransformative tasks where error may be introduced, the error detectionand correction controls in the process, and the data repositories whosequality is to be assessed.

2. Background Description

As companies increasingly adopt information systems that cover a rangeof functional areas, they have electronic access to vast amounts oftransactional data. Increasingly companies are looking developdashboards where a variety of key performance indicators that arecomposed from the transactional data are displayed to assist to businessdecisions. The quality of data contained in these enterprise informationsystems has important consequences, both from the internal perspectiveof making business decisions based on the data as well as the legalobligation to provide accurate reporting to external agencies andstakeholders. As a result, companies spend considerable time and moneyto assess and improve the quality of data in the transactions that flowthrough its information systems and are stored in its repositories.

A considerable body of literature exists on the issue of data qualityassessment from the perspective of auditing a given informationprocessing system. The prior work on data quality management comes fromthe fields of financial accounting and auditing and information systems.

Data quality and control assessment has been studied in accountingliterature since the early 1970s. Most of the studies have approachedreliability assessment with the accounting system viewed as a “blackbox” that transforms data into aggregations of account balancescontained in various ledgers (see, for example, W. R. Knechel, “The useof Quantitative Models in the Review and Evaluation of Internal Control:A Survey and Review”, Journal of Accounting Literature, (Vol. 2), Spring1983:205-219). This approach works well from the perspective of anauditor who is interested in assessing the reliability with which theblack box performs the data transformations. We review this literatureto make note of the key concepts, definitions, and analyses that weadopt and extend in order to develop data quality modeling and analysistechniques at the detailed level of the transformational tasks andprocesses that are contained within the accounting system.

B. E. Cushing in “A Mathematic Approach of the Analysis and Design ofInternal Control Systems” in The Accounting Review 1974, pp. 24-41,developed a mathematic formulation for measuring the reliability for anaccounting system. He used the probability that the system makes noerrors of any kind in its outputs as the system reliability measure. Healso derived a cost measurement by taking into consideration of the costof executing error correction controls and the risk of undetected errorsin the system. It is useful in the sense of evaluating the reliabilityassessment of a given system. However, Cushing's control model takes thesystem structure as given; it does not address any problem from thesystem design perspective. We apply the same basic concepts ofreliability and cost measurement to the problems of evaluating systemreliability for a detailed process model and to design the optimal setof corrective controls with the objective of cost minimization.

S. S. Hamlen in “A Chance-Constrained Mix Integer Programming Model forInternal Control Systems”. The Accounting Review 1980, pp. 578-593,proposed a mixed integer programming model for designing an internalcontrol system. Her model minimizes the cost of controls subject to agiven percentage of quality improvement desired in the output from thesystem. In order to formulate a linear program, the model imposesinstrumental polynomial terms with their respective constraints whichhave the drawback of growing exponentially with the number of terms. Theaccounting system is modeled as a set of controls that can correct a setof error types (which could be errors in various ledgers). We extendHamlen's approach to a more detailed model that identifies error sourceswithin the business process of the accounting system and controls thatmay be selectively applied to these error sources. Our model also allowsus to assess the effect of applying a control to an error source on theresulting probability of errors at all the ledgers that are linked tothat error source. This leads to greater flexibility in selectingcontrols to apply with the potential of better solutions. We also showhow our optimization problem formulation, though more detailed thanHamlen's, can be reduced to a non-exponential series of knapsackproblems without having to convert a non-linear system into a linearone.

Other research in accounting literature focused on probabilisticmodeling and quantitative assessment of accounting information systemreliability. These studies have focused at the accounting system levelmodeling of reliability assessment using probabilistic or deterministicmethods. They treat the transactions streams and transformativeprocesses within the accounting information systems as a black box.Recent studies have begun to develop more detailed models for theassessment of accounting system reliability.

R. B. Lea, S. J. Adams, and R. F. Boykin in “Modeling of the audit riskassessment process at the assertion level within an account balance”,Auditing: A Journal of Practice & Theory 1992 (Vol. 11, Supplement):152-179, discussed the audit risk assessment models at different levelsof detail within accounting systems. They model how risks of error atthe level of the various transaction streams are related to the risk oferror at the account balance level to which they contribute. They notethat the level of tolerable error at the transaction stream level cannotbe assumed to be the same as that for the account balance level. Theirrisk model covers both inherent risk (in the absence of internalcontrols) and control risk. We follow their motivation to decompose anaccount balance to its constituent transaction streams but extend theirpurely additive model to include (a) the volume of transactions in thevarious streams and (b) the probabilistic network structure of thesetransaction streams, identifying the various sources of errors (asrepresented by a process model). This allows us to overcome theassumption made by their model that the errors in the varioustransaction streams are independent.

R. Nado, M. Chams, J. Delisio, and W. Hamscher in “Comet: An Applicationof Model-Based Reasoning to Accounting Systems”, Proceedings of theEighth Innovative Applications of Artificial Intelligence ConferenceAAAI Press (1996) pp. 1482-1490, developed a process model basedreasoning system, which they called “Comet”, for analyzing theeffectiveness of controls. This is one of the earliest attempts todecompose the accounting system structure into the level of tasks thatprocess transactions and implement internal controls. They modeledaccounting systems as a hierarchically structured graph with nodesrepresenting the transaction processing activities and collectionpoints. The potential for failure in each activity is propagated to thecollection points that are the accounts being audited. Controls aremodeled in terms of the probability that they will not cover thefailures. This model can be used to select the key set of controls thatreduce the risk of failure below a threshold. However, the paper doesnot clarify the quantitative model (if any) that is used. It models onlythe probability of failures but ignores the magnitude of error in thesefailures. It also implicitly assumes identical and fixed costs for allcontrols. Our model adopts the basic process modeling conceptsintroduced in this paper and extends them to develop the quantitativeframework described hereinafter. This enables the performance ofrigorous quantitative analysis including Monte Carlo simulation ofinherent and control risk and optimization of control usage based onrisk and cost.

Research on data quality in the information systems literature hasfocused on identifying the important characteristics that define thequality of data (see, for example, Y. Wand and R. Y. Wang, “Anchoringdata quality dimensions in ontological foundations”, Communications ofthe ACM (39:11) (1996), pp. 86-95, and R. Y. Wang, “A ProductProspective on Total Data Quality Management”, Communications of theACM, (41:2) (1998), pp. 58-65). Recently, the management of data qualityand the quality of associated data management processes has beenidentified as a critical issue (see D. Ballou, R. Wang, H. Pazer, and G.Tayi, “Modeling Information Manufacturing Systems to DetermineInformation Product Quality”, Management Science (44:4), April, 1998,pp. 462-484). However, most of the papers describe the criteria for theinformation systems design to improve or achieve good data quality (DQ)or information quality (IQ). To our knowledge, none of the papers havetackled data quality management from the point of view quantitativereliability assessment and optimization, nor did they bring the costs ofquality and quality improvement into the DQ or IQ assessmentconsideration. We consider these issues to be critical from thepractical perspective of design and management of enterprise informationsystems.

Wand and Wang, supra, are amongst the first who studied the data qualityin the context of information systems design. They suggested rigorousdefinitions of data quality dimensions by anchoring them in ontologicalfoundations and showed that such dimensions can provide guidance tosystems designers on data quality issues. They developed a set ofOntological Concepts, and defined Design Deficiencies and Data QualityDimensions. Then they presented the analysis of Dimensions and theImplications to Information Systems Design. Wang, supra, and Ballou etal., supra, developed the Total Data Quality Management methodology(TDQM). TDQM consists of the concepts and the principles of informationquality (IQ) and the information product (IP), and procedures ofinformation management system (IMS) for defining, measuring, analyzing,and improving information products.

L. L. Pipino, Y. W. Lee, and R. Y. Wang, in “Data Quality Assessment”,Communications of the ACM, (45:4), (2002), pp. 211-218, introduced threefunctional forms of data quality: simple ratio, min or max operators,and weighted average. Based on these functional forms, they developedthe illustrative metrics for important data quality dimensions. Finally,they presented an approach that combines the subjective and objectiveassessments of data quality, and demonstrated how the approach can beused effectively in practice.

H. Xu in “Managing accounting information quality: an Australian study”,Managing Accounting Information Quality, (2000), pp. 628-634, developedand tested a model that identifies the critical success factors (CSF)influencing data quality in accounting information systems. He firstproposed a list of factors influencing the data quality of AIS from theliterature, and then conducted pilot case studies, using the findingsfrom the pilot study together with the literature to identify possiblecritical success factors for data quality of accounting informationsystems. He did case studies of accounting information quality inAustralian organizations in practice to test and customize the initialresearch model and compared similarities and differences betweenproposed critical success factors with real-world critical successfactors.

E. M. Pierce in “Assessing Data Quality with Control Matrices”,Comminations of the ACM, (47:2), (2004), pp. 82-86, developed atechnique for information quality management based on the practice fromauditing field: an information product control matrix, to evaluate thereliability of an information product. Pierce defined the components ofthe matrix, and presented a way to link the data problems to the qualitycontrols that should detect and correct these data problems during theinformation manufacturing process.

D. Strong, Y. W. Lee, and R. Wang in “Data Quality in Context”,Communications of the ACM, (40:5), (1997), pp. 58-65, propose adata-consumer perspective for data assessments as opposed to thetraditional intrinsic DQ assessment. They presented a set of DQdimensions that consists of not only the Intrinsic DQ, but AccessibilityDQ, Contextual DQ and Representational DQ. The latter three concernabout the user-task context. They argued that data quality assessmentshould incorporate the task context of users and the processes by whichusers' access and manipulate data to meet their task requirements.

Adopted from Strong et al.'s idea, C. Cappiello, C. Francalanci, and B.Pernici in “Data quality assessment from the user's perspective”,International Workshop on Information Quality in Information Systems,2004, proposed a data quality assessment model that takes intoconsideration user requirements in the assessment phase. In theirmathematical formulation, parameters and matrices to capture the userand user class's preference and requirement are introduced. Their modelshowed how data quality assessment should take into account how userrequirements vary with the accessed service.

SUMMARY OF THE INVENTION

Our invention addresses the issue of data quality management from theperspectives of the owner or the consumer of the information processingsystem and predicting and managing the quality of its data when facedwith anticipated changes in the business environment in which the systemoperates. Such changes could include:

-   -   Changes in the relative volume of transactions arriving from        different input sources. For example, a small but fast-growing        business unit alters the mix of sales transactions over time and        therefore impacts the overall quality of sales data.    -   Changes in the business processes and policies that transform        the data in the transactions. For example, automated systems        replace manual tasks or sections of a process are outsourced.    -   Changes in the business controls that attempt to detect and fix        errors in the transaction. For example, the thresholds that        trigger a control are altered or controls are added or removed        as part of process re-engineering.

This invention provides the modeling and analysis for predicting howthese changes impact data quality. Then, on the basis of this predictiveability, optimization techniques are used for the placement of errorcorrecting controls that meet target quality requirements whileminimizing the cost of operating these controls. This analysis alsocontributes to the development of business “dashboards” that allowdecision-makers to monitor and react to key performance indicators(KPIs) based on aggregation of the transactions being processed. Dataquality estimation in real time provides the accuracy of these KPIs (interms of the probability that a KPI is above or below a given value),which may condition the action undertaken by the decision-maker.

Our approach to modeling data quality takes advantage of the increasingemphasis in many businesses on the formal modeling of business processesand their underlying information processing systems. Although theinitial objective of process modeling is usually for resource planning,and services and workflow design purposes, data quality estimation canbe an important secondary outcome.

A business process model can be used to represent the sources oftransactions entering the information processing system and the varioustasks within the process that manipulate or transform thesetransactions. We associate a subset of these tasks as the potentialerror introduction sources and probabilistically model the rate andmagnitude of various error classes at each such task. We also define theinformation repositories such as accounting ledgers and other databaseswhere the transactions are eventually stored and whose quality needs tobe assessed. A network of links (often with probabilistic branches)connects the transaction sources, error sources, and the informationrepositories.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other objects, aspects and advantages will be betterunderstood from the following detailed description of a preferredembodiment of the invention with reference to the drawings, in which:

FIG. 1 is a block diagram of a process network consisting of transactionsources, error sources and audit targets;

FIG. 2A is a block diagram illustrating preventive controls on an errorsource, and FIG. 2B is a block diagram illustrating feed-forward controlon an error source;

FIG. 3 is a block diagram illustrating a sequence of feed-forwardcontrols at an error source; and

FIG. 4 is an influence diagram of a simple control system.

DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT OF THE INVENTION ProcessModel

A business process model represents the flow of physical items orinformational artifacts through a sequence of tasks and sub-processesthat operate on them. The flow may be controlled by different types of“gateways” that can diverge or converge flows using constructs such asbranches, forks, merges, and joins. These elements form a directed graphwith the tasks and gateways as nodes. The graphs may be cyclic (with theprobability of a cycle being less than one) as well as hierarchical,where one of the nodes could be a sub-process containing its owndirected graph.

We extend the business process modeling framework by adding thefollowing attributes relevant to modeling data quality. Consider abusiness process with T tasks, including all the tasks in itssub-processes. We assign some of these tasks to be transaction sources,error sources, and audit targets as defined next.

A start event or initial task in a process may be assigned to be atransaction source. This is the origination point of a transaction inwhich an error is yet to be introduced. A transaction source ischaracterized by a volume of transaction over a predefined time periodand a random variable signifying the quantitative value of thetransaction. For financial accounting data, this is typically the bookvalue of the transaction.

-   -   Let T_(S)⊂T be the set of transaction sources in the process        model.    -   Let x_(k) be a random variable representing the book value a        transaction originating from the transaction source t_(k)∈T_(S).        Errors could occur when data originating from a transaction        source passes through a subsequent task that is assigned to be        an error source. Error sources are tasks that operate on the        incoming transaction and could introduce errors in them.    -   Let T_(E)⊂T be the set of error sources in the process model.    -   Let P_(i)(ε) be the error incidence probability for error class        ε in the error source t_(i)∈T_(E).        Borrowing from financial accounting practice, we consider three        classes of error:    -   1. Valuation error, which is defined as an error in the        magnitude or value of a valid transaction. This can happen when        a transaction's book value contains the wrong number due to data        entry or mathematical calculation error.    -   Substituting ε=v, let p_(i)(v) be the probability that a        valuation error is introduced at the error source t_(i).    -   Let z_(i) be a random variable representing the taint of the        valuation error. “Taint” is defined as the ratio of the error        magnitude to the book value. If a valuation error is introduced        at the error source t_(i), the magnitude of that error in the        book value is defined as:

e _(i) ^(v) =z ₁ ·x ₁,  (1)

where x_(i) is the observable book value of a transaction at errorsource t_(i) and e₁ ^(v) is the random discrepancy between this hookvalue and the true value of the transaction, known as its audit value.

2. Existence error is defined as the introduction of spurioustransaction entries at the error source. This can happen if the task atthe error source erroneously introduces a new or duplicate transactioninto the business process or fails to follow a business rule that callsfor the cancellation or rejection of a real transaction.

-   -   Substituting ε=e, let p_(i)(e) be the probability that an        existence error is introduced at the error source t_(i).    -   Let x_(i) ^(e) be the random variable for the book value of the        spurious transaction.    -   If an existence error is introduced at the error source i, the        magnitude of that error in the book value is defined as:

e_(i) ^(e)=x_(i) ^(e).  (2)

3. Completeness error occurs when a valid transaction is lost or goesmissing at the error source. This can happen for example when a validtransaction is erroneously deleted or canceled or if there is a failureto create a new data record as required by a business rule at the task.

-   -   Substituting ε=c, let p_(i)(c) be the probability that a        completeness error is introduced at the error source t_(i).    -   If a completeness error is introduced at the error source i, the        magnitude of that error in the book value is defined as:

e_(i) ^(e)=x_(i).   (3)

From the above definitions of the three error classes, note that anerror source can introduce only one class of error in any singletransaction.

Audit targets are repositories in the business process wheretransactions can be stored and retrieved. These could be databasescontaining business and financial data that is used by the company inits decision-making and evaluation of its strategy, or used to generatequarterly and annual financial reports to external parties such asshareholders and regulatory agencies.

-   -   Let T_(A)⊂T be the set of audit targets in the process model        (where we model repositories as tasks).    -   Let X_(j) be the set of transactions in an audit target        t_(j)∈T_(A) and X_(j) ^(ε) be the subset of transactions        containing error of class ε.    -   Let x_(j) be the book value of a transaction in X_(j) and e_(j)        ^(ε) be the magnitude of an erroneous transaction in X_(j) ^(ε).    -   As described in more detail below, we consider three mutually        exclusive classes of error: valuation, existence, and        completeness, denoted by the set [v,e,c]. Let E_(j) ⊂[v,e,c] be        the subset of error classes of interest in the audit target.        Our objective of data quality assessment is to quantify the        error in these repositories according to various error metrics.

-   1. Rate One: Error Incidence, is the ratio of the number of    erroneous transactions of error class ε to the total number of    transactions:

$\begin{matrix}{{R\; 1_{j}^{ɛ}} = {\frac{X_{j}^{ɛ}}{X_{j}}.}} & (4)\end{matrix}$

-   2. Rate Two: Proportion of net monetary error, is the ratio of total    monetary error over all erroneous transactions of error class ε to    the total book value over all transactions:

$\begin{matrix}{{R\; 2_{j}^{ɛ}} = {\frac{\sum\limits_{x_{j}^{ɛ}}e_{j}^{ɛ}}{\sum\limits_{x_{j}}x_{j}}.}} & (5)\end{matrix}$

The proportion of net monetary error can be decomposed into the twofollowing rates:3. Rate Three: Proportion of dollar unit in error or tainting, is theratio of the total monetary error over all erroneous transactions oferror class ε to the total book value over the same set of erroneoustransactions:

$\begin{matrix}{{R\; 3_{j}^{ɛ}} = {\frac{\sum\limits_{x_{j}^{ɛ}}e_{j}^{ɛ}}{\sum\limits_{x_{j}^{ɛ}}x_{j}}.}} & (6)\end{matrix}$

-   4. Rate Four: Proportion of dollar units containing error, is the    ratio of the total book value of all erroneous transactions of error    class ε to the total book value over all the transactions:

$\begin{matrix}{{R\; 4_{j}^{ɛ}} = {\frac{\sum\limits_{x_{j}^{ɛ}}x_{j}}{\sum\limits_{x_{j}}x_{j}}.}} & (7)\end{matrix}$

The transactions sources, error sources and audit targets are connectedto each other by a network of links and gateways. Gateways are definedas the means by which (a) the output from a single task diverges intothe inputs of multiple tasks or (b) the outputs from multiple tasksconverge into the input of a single task. The following types ofgateways are common in a process network:1. Branch: The branch gateway sends the output of a single task to theinput of one out of multiple alternative tasks. The branching decisionis probabilistic (either directly specified or derived from otherbranching criteria).2. Merge: The merge gateway allows the output of multiple tasks to feedinto the input of a single task which is performed when it receives aninput from any one of the tasks being merged.3. Fork: The fork gateway sends the output of a single task to theinputs of multiple tasks at the same time, resulting in the creation ofparallel streams of task activity.4. Join: The join gateway allows the output of multiple tasks to feedinto the input of a single task which is performed only when it receivesinput from all of the tasks being joined. This is usually present tosynchronize the parallel task activities created as a result of a forkupstream in the process network.

We can traverse a process network with the objective of identifying thefollowing parameters that link transaction sources to error sources, anderror sources to audit targets:

-   -   Let V_(kl) be the volume of transactions that flow from a        transaction source t_(k) to a task t_(i) designated as an error        source.    -   Let P_(ij) be the probability that a transaction that flows        through an error source t_(i) will subsequently be stored in an        audit target repository t_(j).

FIG. 1 shows a network diagram linking the transaction sources, errorsources, and audit targets. The dashed links between any two nodesdenote a (possibly null) set of tasks and gateways (hidden in thefigure) that intermediate the flow of transactions between the two nodesin the direction shown. By definition, these hidden tasks cannot betransaction sources, error sources, or audit targets.

As shown by the figure, an error introduced at an error source may bestored in several audit targets. Also, a single audit target may containerrors introduced at multiple error sources. With the data qualityattributes defined above and the network interconnections depicted inFIG. 1, the propagation of transactions and their errors can now becalculated. The volume of transactions, V_(i), reaching error sourcet_(i)∈T_(E) from all transactions sources t_(k)∈·T_(S):

$\begin{matrix}{V_{i} = {\sum\limits_{t_{k} \in T_{S}}{V_{ki}.}}} & (8)\end{matrix}$

The book value, x_(i), of a transaction reaching error source, t_(i):

$\begin{matrix}{x_{i} = {\frac{\sum\limits_{t_{k} \in T_{S}}{x_{k} \cdot V_{ki}}}{V_{i}}.}} & (9)\end{matrix}$

The magnitude of error, e_(i) ^(ε), of error class introduced by errorsource t_(i) is given by Equations (1), (2) and (3) for valuation,existence and completeness errors respectively.

The transactions passing through error sources propagate to audittargets based on the probability P_(ij) which is determined from theprocess network. The aggregation of all transactions from all errorsources results in X_(j) defined above, the set of transactions in anaudit target, t_(j)∈T_(A). The subset of these transactions containingerrors depends on the error incidence probability p_(i)(ε) for eacherror source and the volume of transactions flowing through it.

At each audit target, t_(j)∈T_(A), we calculate the set of error ratesdefined above, corresponding to each of the error classes. Let ∈ε[v,e,c]denote the class of error for which we calculate the error rates.

$\begin{matrix}{{{R\; 1_{j}^{ɛ}} = {\frac{X_{j}^{ɛ}}{X_{j}} = \frac{\sum\limits_{t_{i} \in T_{E}}{V_{i} \cdot {p_{i}(ɛ)} \cdot P_{ij}}}{\sum\limits_{t_{i} \in T_{E}}{V_{i} \cdot P_{ij}}}}},} & (10) \\{{{R\; 2_{j}^{ɛ}} = {\frac{\sum\limits_{x_{j}^{ɛ}}e_{j}^{ɛ}}{\sum\limits_{x_{j}}x_{j}} = \frac{\sum\limits_{t_{i} \in T_{E}}{V_{i} \cdot e_{i}^{ɛ} \cdot {p_{i}(ɛ)} \cdot P_{ij}}}{\sum\limits_{t_{i} \in T_{E}}{V_{i} \cdot x_{i} \cdot P_{ij}}}}},} & (11) \\{{{R\; 3_{j}^{ɛ}} = {\frac{\sum\limits_{x_{j}^{ɛ}}e_{j}^{ɛ}}{\sum\limits_{x_{j}}x_{j}} = \frac{\sum\limits_{t_{i} \in T_{E}}{V_{i} \cdot e_{i}^{ɛ} \cdot {p_{i}(ɛ)} \cdot P_{ij}}}{\sum\limits_{t_{i} \in T_{E}}{V_{i} \cdot x_{i} \cdot P_{ij}}}}},} & (12) \\{{R\; 4_{j}^{ɛ}} = {\frac{\sum\limits_{x_{j}^{ɛ}}x_{j}}{\sum\limits_{x_{j}}x_{j}} = {\frac{\sum\limits_{t_{i} \in T_{E}}{V_{i} \cdot x_{i} \cdot {p_{i}(ɛ)} \cdot P_{ij}}}{\sum\limits_{t_{i} \in T_{E}}{V_{i} \cdot x_{i} \cdot P_{ij}}}.}}} & (13)\end{matrix}$

These equations calculate error rates for a single error class. Asdescribed above, an error source may introduce up to three classes oferrors: valuation, existence, or completeness. For a given transactionhowever, only a single class of error is possible. Due to this mutualexclusion, the sets of erroneous transactions, X_(j) ^(v), X_(j) ^(c),X_(j) ^(e), have no transactions in common (i.e., their pair-wiseintersections result in null sets). As a result of this property, thecombined error rates for all error classes are:

$\begin{matrix}\begin{matrix}{{R\; 1_{j}} = {\sum\limits_{ɛ \in E_{j}}{R\; 1_{j}^{ɛ}}}} \\{= \frac{\sum\limits_{ɛ \in E_{i}}{\sum\limits_{t_{i} \in T_{E}}{V_{i} \cdot {p_{i}(ɛ)} \cdot P_{ij}}}}{\sum\limits_{t_{i} \in T_{E}}{V_{i} \cdot P_{ij}}}} \\{= \frac{\sum\limits_{t_{i} \in T_{E}}( {{V_{i} \cdot P_{ij}}{\sum\limits_{ɛ \in E_{i}}{p_{i}(ɛ)}}} )}{\sum\limits_{t_{i} \in T_{E}}{V_{i} \cdot P_{ij}}}}\end{matrix} & (14) \\\begin{matrix}{{R\; 2_{j}} = {\sum\limits_{ɛ \in E_{i}}{R\; 2_{j}^{ɛ}}}} \\{= \frac{\sum\limits_{ɛ \in E_{i}}{\sum\limits_{t_{i} \in T_{E}}{V_{i} \cdot e_{i}^{ɛ} \cdot {p_{i}(ɛ)} \cdot P_{ij}}}}{\sum\limits_{t_{i} \in T_{E}}{V_{i} \cdot x_{i} \cdot P_{ij}}}} \\{= \frac{\sum\limits_{t_{i} \in T_{E}}( {{V_{i} \cdot P_{ij}}{\sum\limits_{ɛ \in E_{i}}{e_{i}^{ɛ} \cdot {p_{i}(ɛ)}}}} )}{\sum\limits_{t_{i} \in T_{E}}{V_{i} \cdot x_{i} \cdot P_{ij}}}}\end{matrix} & (15)\end{matrix}$

where the set E_(j) ⊂[v,e,c] consists of the error classes of interestat the audit target.

These metrics can be directly calculated if point estimates (or meansonly) are given for the input random variables (such as the transactionbook values x_(k) and the taints z_(i)). If instead, probabilitydistributions are specified for the random variables, Monte Carlosimulation can be done to arrive at probability distributions for theoutputs.

Cost of error arises from the failure of reduce or correct errors thataccumulate at the audit targets of a transaction process. The cost mayarise due to the additional cost or losses incurred because of operatingthe business with incorrect information (for example poor targeting ofpotential customers due to erroneous sales data). The cost could also bein the form of penalties assessed by regulatory and legal agencies dueto misstatements made as a result of incorrect data in financialledgers.

Let ω₁ be the unit cost per erroneous transaction.

Let ω₂ be the unit cost per unit of monetary error.

Then, the total cost due to the number of erroneous transactions foraudit target t_(j) can be obtained as follows, applying Equation (14):

$\quad\begin{matrix}\begin{matrix}{\Omega_{1,j} = {R\; {1_{j} \cdot \omega_{1} \cdot {X_{j}}}}} \\{= {\omega_{1} \cdot {\sum\limits_{t_{j} \in T_{i}}( {{V_{i} \cdot P_{ij}}{\sum\limits_{ɛ \in E_{j}}{p_{i}(ɛ)}}} )}}}\end{matrix} & (16)\end{matrix}$

The total cost due to the magnitude of monetary error for audit targett_(j) can be obtained as follows, applying Equation (15):

$\quad\begin{matrix}\begin{matrix}{\Omega_{2,j} = {R\; {2_{j} \cdot \omega_{2} \cdot {\sum\limits_{x_{j}}x_{j}}}}} \\{= {\omega_{2} \cdot {\sum\limits_{t_{j} \in T_{F}}( {{V_{i} \cdot P_{ij}}{\sum\limits_{ɛ \in E_{j}}{e_{i}^{ɛ} \cdot {p_{i}(ɛ)}}}} )}}}\end{matrix} & (17)\end{matrix}$

The total cost across all audit targets t_(j)∈T_(A) is:

$\begin{matrix}{\Omega = {\sum\limits_{t_{j} \in T_{A}}{( {\Omega_{1,j} + \Omega_{2,j}} ).}}} & (18)\end{matrix}$

The set of equations introduced in this section enables the assessmentof data quality at an audit target both in terms of error rates andcost. This assessment takes into account the structure of the businessprocess and the location of transaction sources and error sources withinit. Process owners can use this assessment to quantify the impact ofchanges in process structure or transaction volumes on the quality ofdata being stored. In auditing terminology, this level of analysisestimates the inherent risk of the accounting system. In the nextsection, we begin to estimate the effect of applying error detection andcorrection controls in order to reduce error rates and costs.

Control Model

Businesses implement internal control systems to reduce the incidence oferrors in its business processes. Controls may be implemented either toprevent errors from being introduced or to monitor for and detect errorsafter they have been generated at error sources. In the latter case, thecontrol could attempt to correct the errors as they are detected(feed-forward control) or to report them so that the error-producingaction may be eventually corrected (feedback control) (see B. E.Cushing, “A Further Note on the Mathematic Approach to InternalControl”, The Accounting Review, Vol. 50, No. 1, 1975, pp. 141-154).

For our model, we consider the controls that have a direct impact onreducing the number of erroneous transactions introduced at an errorsource. This includes preventive and feed-forward controls but excludesfeedback controls because they lack the direct corrective action onerroneous transactions. FIGS. 2A and 2B show how these control typesinteract with an error source and alter its probability of introducingan error from p(ε) to p(ε_(c)). More particularly, FIG. 2A shows theimpact of preventive control on an error source, and FIG. 2B shows theimpact of feed-forward control on an error source. Note that thecontrols may only impact the probability of an error, not its taint.

An error source may have a sequence of feed-forward controls associatedwith it to monitor, detect, and fix errors that may be introduced by theerror source or by any of the intervening controls. This is shown inFIG. 3. The figure also depicts the possibility that not all thetransactions that leave an error source may be sent to a control. Arandom sampling or a business rule may be used to select the subset oftransactions that are sent to each control.

To develop the mathematical formula for calculating, p(ε_(K)), theprobability of error after a sequence of controls K is applied, let usconsider the simplest case of one error source and one feed-forwardcontrol. There are four variables in the system to describe the state ofthis control system:

-   -   the error status E=(ε, ε),    -   the control signaling status C_(s)=(c_(s), c _(s)),    -   the control fixing status C_(f)=(c_(f), c _(f)), and    -   the error status after the application of the control, E_(c)=(ε,        ε).        There are eight possible states in the control system as        described in the table below, along with the resulting impact on        the error status E_(e) after the application of the control:

TABLE 1 Control System States State E C_(s) C_(f) Description E_(c) 1 εc_(s) c_(f) An error exists, the control signals the error, ε and fixesit. 2 ε c_(s) c _(f) An error exists, the control signals the error, εand does not fix it. 3 ε c _(s) c_(f) An error exists, the control doesnot signal ε the error, but somehow takes an action of fixing it. 4 ε c_(s) c _(f) An error exists, the control does not signal ε the error,nor fixes it. 5 ε c _(s) c_(f) An error does not exist, the control doesnot ε signal the error, but somehow takes an action of error “fixing”. 6ε c _(s) c _(f) An error does not exist; the control does not ε signalthe error, nor fixes it. 7 ε c_(s) c_(f) An error does not exist; thecontrol signals ε an error, and takes an action of error “fixing”. 8 εc_(s) c _(f) An error does not exist; the control signals ε an error,but no fixing action.We define the following exogenous attributes of a feed-forward controlthat represent the effectiveness of the control (we show later thatpreventive controls can be formulated as a special case):

-   -   p(c_(s)|ε): the probability that the control signals an error ε        in an error source, given that the error ε exists.    -   p(c_(s)| ε): the probability that the control signals an error ε        in an error source, given that the error ε does not exist        (contra factual).    -   p(c_(f)|c_(s)): the probability that the control takes an action        of error fixing, given that it signals an error ε in an error        source.    -   p(c_(f)| c _(s)): the probability that the control takes an        action of error fixing, given that it does not signal an error ε        in an error source (contra factual).        The influence diagram of this control system is shown in FIG. 4.        The diagram shows that if the status of C_(s) is known, then the        status of C_(f) is independent of the status of E, i.e., C_(f)        and E are conditionally independent given:

p(E,C _(f) |C _(s))=p(E|C _(s))·p(C _(f) |C _(s))  (19)

From this conditional independence, we have:

$\quad\begin{matrix}\begin{matrix}{{p( { C_{f} \middle| C_{s} ,E} )} = \frac{p( {E, C_{f} \middle| C_{s} } )}{p( E \middle| C_{s} )}} \\{= \frac{{p( E \middle| C_{s} )} \cdot {p( C_{f} \middle| C_{s} )}}{p( E \middle| C_{s} )}} \\{= {p( C_{f} \middle| C_{s} )}}\end{matrix} & (20)\end{matrix}$

Using this, we derive the probability of any state in the control systemas follows:

$\quad\begin{matrix}\begin{matrix}{{p( {E,C_{s},C_{f}} )} = {{p( {C_{f}, C_{s} \middle| E } )} \cdot {p(E)}}} \\{= {{p( { C_{f} \middle| C_{s} ,E} )} \cdot {p( C_{s} \middle| E )} \cdot {p(E)}}} \\{= {{p( C_{f} \middle| C_{s} )} \cdot {p( C_{s} \middle| E )} \cdot {p(E)}}}\end{matrix} & (21)\end{matrix}$

We assume the following for feed-forward controls:

-   -   If an control does not signal an error, there will never be an        action of fixing an error, i.e., p(c_(f)| c _(s))=0 and p( c        _(f)| c _(s))=1.    -   If an control does signal an error, there will always be an        action of fixing an error, i.e., p(c_(f)|c_(s))=1 and p( c        _(f)|c_(s))=0.        These assumptions are always true for preventive controls along        with p(c_(s)| ε)=0. That is, we formulate a preventive control        as a special case of the feed-forward control where p(c_(s)|ε)        is the only parameter that can have a value between 0 and 1.        This parameter represents the effectiveness of the control in        preventing an error from being generated by the error source.

Under these assumptions, Equation (21) reduces to the following for eachof the eight states in the control system:

p(ε,c _(s) ,c _(f))=p(c _(s)|ε)·p)(ε)

p(ε,c _(s) , c _(f))=0

p(ε, c _(s) , c _(f))=0

p(ε, c _(s) , c _(f))=p( c _(s)|ε)·p(ε)

p( ε, c _(s) ,c _(f))=0

p( ε, c _(s) , c _(f))=p( c ,| ε)·p( ε)

p( ε,c _(s) , c _(f))=p(c ,| ε)·p( ε)

p( ε,c _(s) , c _(f))=0

Now we derive p(ε_(c)), the probability of error ε in an error sourceafter a single control c has been applied:

$\quad\begin{matrix}\begin{matrix}{{p( ɛ_{c} )} = {{p( {ɛ,{\overset{\_}{c}}_{s},{\overset{\_}{c}}_{f}} )} + {p( {\overset{\_}{ɛ},c_{s},c_{f}} )} + {p( {\overset{\_}{ɛ},{\overset{\_}{c}}_{s},c_{f}} )} + {p( {ɛ,c_{s},{\overset{\_}{c}}_{f}} )}}} \\{= {{p( {ɛ,{\overset{\_}{c}}_{s},{\overset{\_}{c}}_{f}} )} + {p( {\overset{\_}{ɛ},c_{s},c_{f}} )}}} \\{= {{{p( {\overset{\_}{c}}_{s} \middle| ɛ )} \cdot {p(ɛ)}} + {{p( c_{s} \middle| \overset{\_}{ɛ} )} \cdot {p( \overset{\_}{ɛ} )}}}} \\{= {{{p(ɛ)} \cdot ( {1 - {p( c_{s} \middle| ɛ )}} )} + {( {1 - {p(ɛ)}} ) \cdot {{p( c_{s} \middle| \overset{\_}{ɛ} )}.}}}}\end{matrix} & (22)\end{matrix}$

If the control c is applied only to a fraction y of all the transactionscoming out of the error source, Equation (22) is modified to:

$\quad\begin{matrix}\begin{matrix}{{p( ɛ_{c} )} = {{y \cdot \lbrack {{{p(ɛ)} \cdot ( {1 - {p( c_{s} \middle| ɛ )}} )} + {( {1 - {p(ɛ)}} ) \cdot {p( c_{s} \middle| \overset{\_}{ɛ} )}}} \rbrack} +}} \\{{( {1 - y} ) \cdot {p(ɛ)}}} \\{= {{{p(ɛ)} \cdot ( {1 - {{yp}( c_{s} \middle| ɛ )}} )} + {( {1 - {p(ɛ)}} ) \cdot ( {{yp}( c_{s} \middle| \overset{\_}{ɛ} )} }}}\end{matrix} & (23)\end{matrix}$

Next, we consider p(ε_(K)), the probability of error ε after theapplication of a sequence of controls K to an error source, as shown onFIG. 3. Let K=[c_(j)]|j=1 . . . J, where c_(j) is the j-th control inthe sequence. Then, the probability of error after the application ofthe j-th control is:

p(ε_(c,j))=p(ε_(c,j-1))·(1−y _(j) p(c _(s,j)|ε))+(1−p(ε_(s,j-1)))·(y_(j) p(c _(c,j)| ε))  (24)

where, the j subscript in the other variables denote the respectivevariables for the j-th control.

Equation (24) can be iteratively calculated to computep(ε_(c))=p(ε_(c,J)) starting at p(ε_(c,0))=p(ε). This quantifies theeffect of applying a regime of controls K to a single error source.Using the error propagation formulation described above, we can nowassess the impact of the controls on the error rates and cost of errorat the audit targets in the business process. In auditing terminology,this level of analysis estimates the control risk in the accountingsystem.

The application of controls at an error source incurs a cost. Weconsider this cost to be linearly proportional to the number oftransactions passing through the control. This cost consists of the costto detect if an error exists and the cost to fix the error if found. Letω(c_(s)) be the cost to monitor, detect and signal an error (incurred onall transactions passing through the control) and ω(c_(f)) be the costof fixing each error (incurred only on the transactions deemederroneous). Then, the cost per transaction passing through the controlis:

ω(c)=ω(c _(s))+ω(c _(f))·(p(c _(f) ,c _(x), ε)+p(c _(f) ,c _(s), ε)+p(c_(f) , c _(s), ε)+p(c _(f) , c _(s),ε))

Applying the assumptions for feed-forward controls and Equation (21),

ω(c)=ω(c _(s))+ω(c _(f))·(p(c _(s)| ε)·(1−p(ε))+p(c _(s)|ε)·p(ε)).  (25)

Considering T_(E) error sources and a sequence of controls K, at anerror source t_(t)∈T_(E), we have the total cost of controls in thebusiness process:

$\begin{matrix}{\Omega_{C} = {\sum\limits_{t_{j} \in T_{E}}( {V_{i}{\sum\limits_{c_{j} \in K_{i}}{y_{j}{\omega ( c_{j} )}}}} )}} & (26)\end{matrix}$

where V_(i), as defined in Equation (8), is the volume of transactionsreaching the error source t_(i).

Now we are in a position to formulate optimization problems that tradeoff the cost of controls at the error sources with the cost of error atthe audit targets. This is done in the next section.

Optimization

The business process and control models developed above allow us toformulate the following series of optimization problems.

-   -   For these formulations, we use the following variables: The        overall system reliability (1−R) across all the audit targets,        where R is either

$\sum\limits_{t_{j} \in T_{A}}{R\; 1_{j}}$

as defined by Equation (14) or

$\sum\limits_{t_{j} \in T_{A}}{R\; 2_{j}}$

as defined by Equation (15).

-   -   The total cost of error across all audit targets, Ω as given in        Equation (18).

The total cost of controls in the business process, Ω_(C) as given inEquation (26).

The decision variables y_(j), which is the fraction of transactions aterror source t_(i)∈T_(E) that will be sent to a control c_(j)∈K_(i),where K_(i) is the sequence of controls available for the error sourcet_(i).

Using the above notation, the optimization formulations are as follows:

-   -   1. Maximize the system reliability (1−R), subject to a budget        {circumflex over (Ω)}_(C) for the total control cost in the        business process, i.e., Ω_(C)≦{circumflex over (Ω)}_(C).    -   2. Minimize the control cost Ω_(C), subject to a target system        reliability (1−{circumflex over (R)}), i.e., R≦{circumflex over        (R)}.    -   3. Minimize the cost of error Ω, subject to a budget {circumflex        over (Ω)}_(C) for the total control cost in the business        process, i.e., Ω_(C)≦{circumflex over (Ω)}_(C).    -   4. Minimize the control cost Ω_(C), subject to a budget        {circumflex over (Ω)} for the total cost of error in the        business process, i.e., Ω≦{circumflex over (Ω)}.    -   5. Minimize the total cost in the process (Ω+Ω_(C)).

As a special case with a tractable solution, consider the optimizationproblem 4 above, where the cost of control must be minimized to as tokeep the cost of error in the system below a threshold budget{circumflex over (Ω)}.

We solve this problem by dividing it into two sub-problems. Onesub-problem is at the audit targets stage, where we wish to minimize thetotal cost of error, given sets of controlled error levels and theircorresponding control cost for each error source. The second sub-problemis to come up with these sets at the error sources stage, where we wishto minimize the control cost for a given error level.

For the audit target stage sub-problem, Equation (18) calculates thetotal cost of error across all audit targets Ω, which can be written asfollows, if we consider only a single class of error in our analysis:

$\begin{matrix}{\Omega = {\sum\limits_{t_{j} \in T_{A}}( {\sum\limits_{t_{i} \in T_{E}}{V_{i} \cdot {P_{ij}( {\omega_{1} + {\omega_{2}e_{i}^{ɛ}}} )} \cdot {p_{i}(ɛ)}}} )}} & (27)\end{matrix}$

To meet the Ω≦{circumflex over (Ω)} requirement, we need to applycontrols at one or more error sources to reduce the “posterior” errorrates p(ε_(K)) at some cost. For each error source t_(t), wecharacterize a set of pairs: {(ω_(i,k),p_(i)(ε_(k) _(i) ))|k_(t)∈{1, 2,. . . K_(i)}}, where ω_(i,k), is the cost of reducing the error level att_(i) to p_(i)(ε_(k) _(i) ). As described below for the secondsub-problem, where we optimize the cost of controls at the error sourcesstage, k_(i), is a control strategy that can be applied at the errorsource t_(i). Table 2 below shows the different levels controls and theassociated cost and reliability levels.

TABLE 2 The different error levels (by applying controls) and associatedcost of control $\begin{matrix}{{error}\mspace{14mu} {source}\mspace{14mu} 1} \\\overset{}{\begin{matrix}( {\omega_{1,1},{p_{1}( ɛ_{1} )}} ) \\( {\omega_{1,2},{p_{1}( ɛ_{2} )}} ) \\\vdots \\( {\omega_{1,{K1}},{p_{1}( ɛ_{K1} )}} )\end{matrix}}\end{matrix}{\quad\quad}$ $\begin{matrix}{{error}\mspace{14mu} {source}\mspace{14mu} 2} \\\overset{}{\begin{matrix}( {\omega_{2,1},{p_{2}( ɛ_{1} )}} ) \\( {\omega_{2,2},{p_{2}( ɛ_{2} )}} ) \\\vdots \\( {\omega_{2,{K2}},{p_{2}( ɛ_{K2} )}} )\end{matrix}}\end{matrix}\quad$ ${\begin{matrix}{{error}\mspace{14mu} {source}\mspace{14mu} 3} \\\overset{}{\begin{matrix}( {\omega_{3,1},{p_{3}( ɛ_{1} )}} ) \\( {\omega_{3,2},{p_{3}( ɛ_{2} )}} ) \\\vdots \\( {\omega_{3,{K3}},{p_{3}( ɛ_{K3} )}} )\end{matrix}}\end{matrix}\ldots},\ldots$ $\begin{matrix}{{error}\mspace{14mu} {source}\mspace{11mu} I} \\\overset{}{\begin{matrix}( {\omega_{I,1},{p_{I}( ɛ_{1} )}} ) \\( {\omega_{I,2},{p_{I}( ɛ_{2} )}} ) \\\vdots \\( {\omega_{I,{K1}},{p_{I}( ɛ_{K1} )}} )\end{matrix}}\end{matrix}\quad$The objective here is to pick an appropriate level of control at eacherror source so as to keep the system level cost of error below thethreshold budget {circumflex over (Ω)}. This can be written as follows:

$\begin{matrix}{{{\min {\sum\limits_{t_{j} \in T_{i}}{\overset{K_{i}}{\sum\limits_{k \in 1}}( {\omega_{i,k_{i}} \cdot z_{i,k_{i}}} )}}}{s.t.\mspace{14mu} \Omega} = {{\sum\limits_{t_{i} \in T_{A}}( {\sum\limits_{t_{i} \in T_{e}}{V_{i} \cdot {P_{ij}( {\omega_{1} + {\omega_{2}e_{i}^{ɛ}}} )} \cdot ( {\sum\limits_{k_{i} = 1}^{K_{i}}\; {z_{i,k_{i}} \cdot {p_{i}( ɛ_{k_{i}} )}}} )}} )} \leq \hat{\Omega}}}{{{\sum\limits_{k_{i} = 1}^{K_{i}}z_{i,k_{i}}} \leq 1},{z_{i,k_{i}} = \{ \begin{matrix}{1,} & {{if}\mspace{14mu} {control}\mspace{14mu} {level}\mspace{14mu} k_{i}\mspace{14mu} {is}\mspace{14mu} {chosen}\mspace{14mu} {for}\mspace{14mu} {error}\mspace{14mu} {source}\mspace{14mu} i} \\{0,} & {{if}\mspace{14mu} {control}\mspace{14mu} {level}\mspace{14mu} k_{i}{\mspace{11mu} \;}{is}\mspace{14mu} {not}\mspace{14mu} {chosen}\mspace{14mu} {for}\mspace{14mu} {error}\mspace{14mu} {source}\mspace{14mu} i}\end{matrix} }}} & (28)\end{matrix}$

The decision variable is z_(i,k) _(i) , k_(i)∈{1, 2, . . .K_(i)}·z_(i,k) _(i) , is a binary variable, which takes the value of 1if the pair (ω_(i,k) _(i) , p_(i)(ε_(k) _(i) )) is chosen for the errorsource i. The constraint

${{\sum\limits_{k_{i} = 1}^{K_{i}}z_{i,k_{i}}} \leq 1},$

implies that only one reliability level for each error source i can bechosen. Recognizing this problem as the multiple choice knapsack problem(see, for example, S. Martello and P. Toth, Knapsack Problems,Algorithms and Computer Implementations, John Wiley and Sons Ltd.,England, 1990) which can be solved by dynamic programming in O(K×W)where K is the total number of levels across all error sources and W isrelated to the accuracy with which {circumflex over (Ω)} needs to beachieved.

Next, we develop a control model to compute the minimum cost controlstrategy for each level at each error source. Although this implies theneed to solve an optimization model to compute each (cost, error level)pair, we will show that this optimization model is a knapsack problemwhich is relatively easy to solve.

For the sub-problem at the level of the error sources, our objective isto come up with a set of (ω_(i,k) _(i) , p_(i)(ε_(k) _(i) )) pairs foreach error source. In doing so, we wish to minimize the cost ω_(i,k)_(i) of reducing the error level at error source t_(i) to p_(i)(ε_(k)_(i) ).

Equation (24) provides the means for iteratively calculating p_(i)(ε_(k)_(i) ) for a given set of controls K_(i) at error source t_(i) and acontrol strategy defined by the fraction of transactions y_(j), j∈{1, 2,. . . , |K_(i)|}, reaching each control c_(j)∈K_(i). If we canreasonably assume that a control attempting to fix a non-error will notintroduce an error, i.e., the states 5 and 7 in Table 1, E_(c)= ε. Withthis the error incidence rate p(ε_(K)) simplifies to:

$\begin{matrix}{{p( ɛ_{K_{i}} )} = {{p(ɛ)} \cdot {\prod\limits_{j = 1}^{K_{t}}( {{1 - {y_{j}{{p( c_{s,j} \middle| ɛ )}.\ln}\; {p( ɛ_{K_{i}} )}}} = {{\ln \; {p(ɛ)}} + {\sum\limits_{j = 1}^{K_{i}}{\ln ( {1 - {y_{j}{p( c_{s,j} \middle| ɛ )}}} )}}}} }}} & (29)\end{matrix}$

Observe that we have linearized the expression for p(ε_(K) _(i) ) usinglogarithms. This suggests that the sequence in which the controlsc_(j)∈K_(i) are applied is inconsequential. So a simple optimizationformulation for a single error source with multiple controls is asfollows: Given c_(j)∈K_(i) control units and a the target error levelp(ε_(k) _(i) ), find the optimal control strategy k_(i), specified interms of y_(j), j∈{1, 2, . . . , |K_(i)|}, that minimizes the controlcost:

$\begin{matrix}{{\min {\sum\limits_{j = 1}^{K}( {{\omega ( c_{j} )} \cdot y_{j}} )}}{{{{s.t.\mspace{14mu} \ln}\; {p(ɛ)}} + {\sum\limits_{j = 1}^{K_{i}}{\ln ( {1 - {y_{j}{p( c_{s,j} \middle| ɛ )}}} )}}} \leq {\ln \; {p( ɛ_{k_{i}} )}}}{y_{j} \in \lbrack {0,1} \rbrack}} & (30)\end{matrix}$

where ω(c_(j)) is the per-transaction cost of applying the jth controlto the error source as defined in Equation (25). Although we haveassumed (implicitly by making y_(j) binary,) that the controls areapplied to all the transactions or none, this can be easily relaxed toallow the control of a fraction of the transactions. Notice that theabove problem is a knapsack problem that can be solved by dynamicprogramming (see, again. Martello and Toth 1990, supra) in O(J×R) whereJ is the number of controls and R is a number based on the accuracydesired of p(ε_(k) _(i) ).

Noting from Equation (29) that the sequence of applying controls doesnot impact the probability of error after the application of controls,we construct a simple algorithm that can find the optimal controlstrategy k_(i) for a given target error level p(ε_(k) _(i) ). This isshown in Table 3. We select the control with the highestcost-effectiveness ratio and apply it to all the transactions in theerror source. If the resulting error level is still higher than thetarget, we apply the control with the next highest cost-effectivenessratio. When the error level falls below the target, we adjust thesampling fraction y_(j) of the last selected to so as to achieve thetarget error level. Thus, the sampling fractions of all controls will be1 or 0 with the exception of one control, whose sampling fraction willbe in [0-1].

TABLE 3 Algorithm for Control Strategy Selection Given target errorlevel, p(ε_(k) _(i) ) Candidate control set K = {C₁, C₂, . . . C_(j)},Solution set [y_(j)]|j ε {1,2, . . . , |K|} = [0] Set P = p(ε) 1.${{Calculate}\mspace{14mu} {the}\mspace{14mu} {cost}\text{-}{effectiveness}\mspace{14mu} {ratio}},\frac{p( c_{s,j} \middle| ɛ )}{\omega ( c_{j} )},$for each candidate control unit in K 2. Choose the control that has thehighest value${j\text{*}} = {\max\limits_{c_{j}{\varepsilon K}}( \frac{p( c_{s,j} \middle| ɛ )}{\omega ( c_{j} )} )}$3. Update P = P · (1 − p(c_(s,j*)|ε)) 4. if P > p(ε_(k) _(i) ), sety_(j*) = 1; take C_(j*) off the candidate list K, if K ≠ φ go to step 2else terminate the procedure with failure else${{set}\mspace{14mu} y_{j\text{*}}} = \frac{P - {{p( ɛ_{k_{i}} )}( {1 - {p( c_{s,{j\text{*}}} \middle| ɛ )}} )}}{P \cdot {p( c_{s,{j\text{*}}} \middle| ɛ )}}$terminate the procedure with success

We have described a framework for the quantitative modeling of dataquality in a business process. We have shown how the model can be usedto make assessments of data quality in a pre-defined process as well asto develop optimal control system designs that meet reliability or costrequirements.

These techniques will be of value to business process owners as well asto evaluators of data quality (such as auditors in case of businessprocesses with financial transactions and accounts). However, the usersof these techniques must adopt a methodology by which the data qualitymodel must be developed and maintained. The methodology comprises of thefollowing steps:

1. Create a model of an existing business process. Various modelingtools are commercially available for this purpose.2. Utilizing the modeling framework developed in the Process Modelsection, identify the transaction sources, error sources, and audittargets.

2.1. For transaction sources, obtain or estimate the volume oftransactions over a given time period (e.g., per day, month, quarter, oretc.) and estimate the transaction book values. This may be a simpleaverage book value or a probability distribution based on historicaltransaction data.

2.2. For error sources, obtain the probability of errors prior to theapplication of any controls. This may be obtained from the logs ofcontrols that already exist. For a new business process or for errorsources that do not have logs of past control activity, an estimationmust be done based on comparable error sources with available data. Thetaint of the error sources must also be obtained from historical logs orotherwise estimated. Note that the taint may be a point estimate or aprobability distribution.

2.3. For audit targets, specify the types of errors of interest and ifany error level requirements exist for them.

3. Run the error propagation analysis described in the Process Modelsection to estimate error rates and cost of error at the audit targets.For a model with probability distributions, a Monte Carlo simulation canbe performed to estimate error rates and costs in terms of probabilitydistributions. The process analyst may develop multiple scenarios totest different expectations of future process changes, such as changesin transaction volumes and business process topology and policies.4. Utilize the control systems model developed in the Control Modelsection to associate error sources with a set of controls. These may beexisting or available controls. For each control, estimate its errordetection and correction effectiveness, as defined by the probabilitiesp(c_(s)|ε) and p(c_(s)| ε). This data is available if the controls areperiodically subject to internal or external auditing, where they areevaluated with test data with known errors. The cost of controls can beestimated from the time spent on each control to search for and then fixerrors.5. Analyze the impact of selected controls using the assessmenttechnique described in the Control Model section. The process analystmay run multiple scenarios with different control selections as well asthe scenarios developed in step 3 above. The cost of the selectedcontrols can be compared with the reliability level or cost of error atthe audit targets.6. When manual search for the optimal control design is intractable, theoptimization techniques shown in step 5 are applicable. Here, we assumethat each error source has a set of potential controls and the problemis to select the fraction of the total transactions to send to each.

Although our model and analyses have been motivated by the types oftransaction errors and error correction controls in the accounting andauditing domain, it can extend to other domains and definitions of dataquality. For example, we can consider that “error” sources introduceuncertainty about the data in a transaction rather than mistakes.Sources of uncertainty could be prices of raw material, customer demand,product development times, service delivery times, etc. We can adapt theerror propagation techniques of this invention to propagate theseuncertainties to the data repositories. We can also then consider theanalogues of “controls” that may reduce these uncertainties, but at acost. For example, uncertainties about raw material prices can bereduced by establishing long-term contracts or hedging with options.Variability in delivery times may be reduced by automating processes.These uncertainty reduction actions come at a cost and we can trade offthese costs with the consequent level or cost of the uncertainty in thedata repositories.

In conclusion, our invention contributes to the analysis of data qualityby incorporating a business process framework for the assessment andoptimization of data quality. This invention applies not only to theliterature and practice of financial accounting and auditing, but alsoto business decision-support systems.

While the invention has been described in terms of a single preferredembodiment, those skilled in the art will recognize that the inventioncan be practiced with modification within the spirit and scope of theappended claims.

1. A data quality management method comprising the steps of: creating amodel of a new or existing business process; utilizing a modelingframework, identifying transaction sources, error sources, and audittargets; running error propagation analysis to estimate error rates andcost of error at the audit targets; utilizing a control systems model toassociate error sources with a set of controls; and analyzing an impactof selected controls using an assessment technique.
 2. The data qualitymanagement method recited in claim 1, further comprising the steps fortransaction sources of obtaining or estimating a volume of transactionsover a given time period and estimating transaction book values.
 3. Thedata quality management method recited in claim 2, wherein estimatingtransaction book values is based on a simple average book value or aprobability distribution based on historical transaction data.
 4. Thedata quality management method recited in claim 1, further comprisingthe steps for error sources of obtaining a probability of errors priorto application of any controls and a taint of the error sources.
 5. Thedata quality management method recited in claim 4, wherein theprobability of errors and the taint of the error sources are obtainedfrom logs of controls that already exist.
 6. The data quality managementmethod recited in claim 4, wherein for a new business process or forerror sources that do not have logs of past control activity, anestimation is done based on comparable error sources with availabledata.
 7. The data quality management method recited in claim 1, furthercomprising the steps for audit targets of specifying types of errors ofinterest and if any error level requirements exist for them.
 8. The dataquality management method recited in claim 1, further comprising thestep for a model with probability distributions of performing a MonteCarlo simulation to estimate error rates and costs in terms ofprobability distributions.
 9. The data quality management method recitedin claim 1, further comprising the step for each control of estimatingits error detection and correction effectiveness.
 10. The data qualitymanagement method recited in claim 1, further comprising the step ofmaximizing the reliability level at audit targets subject to meeting abudget for a cost of controls.
 11. The data quality management methodrecited in claim 1, further comprising the step of minimizing a cost ofcontrols subject to meeting a minimum reliability level at audittargets.